Storage medium having stored thereon program for adjusting pointing device, and pointing device

ABSTRACT

A pointing device includes a controller for pointing a position on a display surface by imaging an imaging objective such as two markers positioned with a predetermined interval therebetween in a vicinity of the display surface of a monitor. With the pointing device, a width of the display surface and the interval between the markers are obtained to determine an appropriate distance between the controller and the monitor. Further, a current distance between the controller and the monitor (markers) is calculated. Relation between the current distance and the appropriate distance is notified by such as image display or sound output. For example, an image indicating the appropriate distance and an adjustment image varying in size in conjunction with the current distance are displayed on the monitor.

CROSS REFERENCE OF RELATED APPLICATION

The disclosure of Japanese Patent Application No. 2006-312145 isincorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a storage medium having stored thereona program for adjusting a pointing device, as well as to a pointingdevice. In particular, the present invention relates to, for example, astorage medium having stored thereon a program for adjusting a pointingdevice, the program conducting information processing using input dataobtained from an input apparatus provided with an imaging means forimaging a known imaging objective to output to a display apparatus, aswell as to a pointing device.

2. Description of the Related Art

An example of this type of pointing device conventionally employed isdisclosed in the document 1 (Japanese Patent Laying-Open No. 08-38741).According to a shooting game apparatus of the document 1, lightalternately emitted from a pair of light emitting elements issequentially and alternately received with a light receiving elementprovided at a muzzle. The light receiving element outputs a plurality ofdetection signals such as a distance from a display screen and anincident angle of the light from each light emitting element. Based onthe plurality of detection signals, a first calculation circuitcalculates aiming coordinates. On the other hand, a second calculationcircuit constantly recognizes coordinates of a target image in thedisplay screen, and determines whether or not the coordinates of thetarget image substantially matches the aiming coordinates upon operationof a switch.

The technique disclosed in the document 1 is commonly seen in arcadegame machines. In a case of a typical arcade game machine, positions ofthe display apparatus and the light emitting elements may be fixedpreviously in a designing phase. Further, because a gun (gun controller)for games provided with the light receiving elements is also connectedvia a cable to the arcade game machine, a range of movement of the guncontroller is previously limited. Consequently, by previously setting sothat coordinates at a right end of the display apparatus becomes theaiming coordinates when the right end of the display apparatus is aimedwith the gun controller while coordinates at a left end of the displayapparatus becomes the aiming coordinates when the left end of thedisplay apparatus is aimed with the gun controller, it is possible toplay a game using any arcade game machine comfortably to some extent.

However, an attempt of realizing the pointing device as described abovein a home-use game machine encounters a problem of varying useenvironment, i.e. a size of a television screen, a distance between adisplay and the gun controller, and an interval between positions of thelight emitting elements are different for each family. Therefore,determining the correspondence based on a fixed configuration may beappropriate in some cases but inappropriate in different cases. Thelatter case may cause a problem such that the gun controller points adirection (position on the display screen) largely deviated from adirection instructed by a player.

SUMMARY

In view of the above problems, a major object of the present inventionis to provide a novel storage medium having stored thereon a program foradjusting a pointing device, as well as a pointing device.

Further, another object of the present invention is to provide a storagemedium having stored thereon a program for adjusting a pointing device,the program being capable of setting a configuration best suited for ause environment, as well as a pointing device.

Example embodiments of the present invention adopt the followingfeatures. It should be noted that the reference numerals inside theparentheses, supplement, etc. only show one example of a correspondencewith the embodiment to be described later in order to aid theunderstanding of the present invention, and do not limit the presentinvention.

A first invention provides a storage medium having stored thereon aprogram for adjusting executed in a computer of a pointing device forobtaining input data from an input apparatus provided with an imagingmeans for imaging a known imaging objective, the input data being one ofdata of an imaged image obtained by the imaging means and data obtainedby performing a predetermined operation on the obtained data of theimaged image, and performing information processing using the input datato output to a display apparatus. The program for adjusting has thecomputer execute: a display surface width obtaining step, an appropriatedistance determining step, a current distance calculating step, and anotifying step. In the display surface width obtaining step, a maximumwidth of a display surface of the display apparatus is obtained. In theappropriate distance determining step, an appropriate distance betweenthe input apparatus and the display apparatus is determined according tothe maximum width of the display surface obtained in the display surfacewidth obtaining step. In the current distance calculating step, acurrent distance from the input apparatus to the imaging objective iscalculated based on the input data. In the notifying step, anotification is made according to relation between the appropriatedistance and the current distance.

In the first invention, the program for adjusting is executed by acomputer (36) of a pointing device (12). With this pointing device, itis possible to obtain an input by imaging a known imaging objective (340m and 340 n) with an imaging means (98) of an input apparatus (22). Thesize, for example, of the imaging objective is previously determined,that is, made known. Further, in one embodiment, the distance betweentwo markers as the imaging objective is made known. In the pointingdevice, the input data as one of the data of the imaged image obtainedby the imaging means and the data obtained by performing thepredetermined operation on the obtained data of the imaged image isobtained, and the information processing is performed using the inputdata to output the resulting data to the display apparatus (34). Theprogram for adjusting has the computer of such a pointing device executea process of adjusting the distance between the input apparatus and theimaging objective or display apparatus to an appropriate distance.Specifically, in the display surface width obtaining step (S3, S3′, andS105), the maximum width of the display surface (display width D) of thedisplay apparatus is obtained. The maximum width of the display surfacemay be obtained either by, for example, a direct input by the user, oran input of the size of the display apparatus by the user.Alternatively, the maximum width of the display surface may becalculated by obtaining a real distance and a distance on the imagebetween the two markers as the imaging objective. In the appropriatedistance determining step (S5), the appropriate distance between theinput apparatus and the imaging objective or display apparatus isdetermined according to the maximum width of the display surface. Asdescribed later, by calculating a distance L according to relation whenan angle θ (expressed by an equation 2) obtained from positionalrelation when ideally pointing an end of the display screen becomesequal to an angle β (expressed by an equation 3) obtained frompositional relation to be a limit of recognition of the imagingobjective, it is possible to obtain the appropriate distance. Forexample, by previously storing a table showing the appropriate distanceassociated with the maximum width of the display surface of aconventional display apparatus, and by referring to this table, theappropriate distance may be obtained. Alternatively, the appropriatedistance may be calculated according to the above relation forcalculating the appropriate distance. In the current distancecalculating step (S11), the current distance (Lc) between the inputapparatus and the imaging objective is calculated based on the inputdata. In an embodiment described later, because the imaging objectiveare known and the width of the imaged image and the viewing angle of theimaging means are determined, the current distance is calculatedaccording to equations 4 and 5 using the distance of the imagingobjective in imaged image. In a notifying step (S13), the notificationaccording to the relation between the appropriate distance and thecurrent distance is made. The notification with which whether or not thecurrent distance is the appropriate distance may be recognized is madeby a method such as image display, sound output, vibration, or lightoutput.

According to the first invention, it is possible to determine theappropriate distance according to the maximum width of the displaysurface of the display apparatus, and to make the notification accordingto the appropriate distance and the current distance. Therefore, it ispossible to notify the user whether or not the current distance is theappropriate distance.

A second invention provides a storage medium having stored thereon aprogram for adjusting a pointing device according to the firstinvention. In the notifying step, the notification is made by one ofoutputting image display, outputting sound such as a voice sound and asound effect, and having the input apparatus vibrate, according to adifference between the current distance and the appropriate distance.

In the second invention, the image according to the difference betweenthe current distance and the appropriate distance is displayed on thedisplay apparatus, or the sound such as the voice sound or the soundeffect according to the above difference is outputted from soundoutputting devices (34 a and 104), or a vibration device (106) of theinput apparatus is vibrated according to the above difference.Therefore, it is readily possible to notify the difference between thecurrent distance and the appropriate distance, thereby facilitating theadjustment to the appropriate distance.

A third invention provides a storage medium having stored thereon aprogram for adjusting a pointing device according to the firstinvention. In the notifying step, the notification is made bydisplaying, on the display apparatus, a reference image having apredetermined size and an adjustment image varying in size inconjunction with the current distance and becoming the same size as thereference image when the current distance is equal to the appropriatedistance.

In the third invention, a reference image (202) of the predeterminedsize and an adjustment image (204) varying in size in conjunction withthe current distance are displayed. When the current distance becomesthe appropriate distance, the size of the adjustment image becomes thesame as that of the reference image. It is possible to facilitate theadjustment to the appropriate distance by changing the distance from theimaging objective to the input apparatus so as to match the size of theadjustment image with the size of the reference image.

A fourth invention provides a storage medium having stored thereon aprogram for adjusting a pointing device according to the firstinvention. In the known imaging objective is constituted by two markerspositioned with a predetermined distance therebetween measured in afirst scale, the two markers being positioned in a vicinity of apredetermined side of the display surface of the display apparatus sothat a positioning direction of the two markers is in parallel to thepredetermined side. The program for adjusting further has the computerexecute a distance obtaining step of obtaining the predetermineddistance either inputted by a user or previously stored. The displaysurface width obtaining step includes: a marker distance measuring stepof measuring a distance between the two markers in a second scale byvarying an interval in the positioning direction between the two markersin an image displayed on the display surface, according to the input bythe user; and a display surface width calculating step of calculatingthe maximum width of the display surface in the first scale based on aratio between the predetermined distance obtained in the distanceobtaining step and the distance between the markers measured in themarker distance measuring step, as well as based on the maximum width ofthe display surface in the second scale.

In the fourth invention, the two markers as the imaging objective arepositioned in a vicinity of the predetermined side of the displaysurface of the display apparatus so that the two markers is in parallelto the predetermined side. The two markers are positioned with thepredetermined distance therebetween measured in the first scale. In thedistance obtaining step (S1), the predetermined distance between the twomarkers (sensor width S) is obtained. Specifically, the predetermineddistance between the two markers is obtained in the first scale. Thefirst scale is, for example, meters as a unit for measuring a realdistance. The predetermined distance between the two markers is inputtedby the user. Alternatively, when the distance between the two markers isfixed, the previously stored predetermined distance between the twomarkers is obtained. In the marker distance measuring step (S41), thedistance between the two markers (sensor-corresponding width W) in thedisplay surface is measured in the second scale, using an image (200) inwhich the interval in the positioning direction between the two markersvaries according to the user input. The second scale is, for example, isa unit for a digital image (pixel), and it is possible to measure thedistance on the image. In the display surface width calculation step(S37), the maximum width of the display surface (display width D) in thefirst scale is calculated based on a ratio of the predetermined distance(the sensor width S) in the first scale and the distance(sensor-corresponding width W) in the second scale, as well as based onthe maximum width of the display surface (the maximum display width M)in the second scale, according to an equation 1 described later, forexample. As described above, because the maximum width of the displaysurface in the second scale is previously determined and stored, it ispossible to calculate the maximum width of the display surface, byobtaining the predetermined distance between the two markers in thefirst scale, and measuring the interval between positions of the twomakers in the second scale. Therefore, the adjustment can be readilyperformed even if the user does not know the maximum width of thedisplay surface of the display apparatus.

A fifth invention provides a storage medium having stored thereon aprogram for adjusting a pointing device according to the firstinvention. The known imaging objective is constituted by two markerspositioned with a predetermined distance therebetween measured in afirst scale, the two markers being positioned in a vicinity of apredetermined side of the display surface of the display apparatus sothat a positioning direction of the two markers is in parallel to thepredetermined side. The program for adjusting further has the computerexecute a distance obtaining step of obtaining the predetermineddistance either inputted by a user or previously stored. The displaysurface width obtaining step includes: a marker distance measuring stepof measuring a distance between the two markers in a second scale byvarying an interval in the positioning direction between the two markersin an image displayed on the display surface, according to the input bythe user; and a display surface width calculating step of calculatingthe maximum width of the display surface in the first scale based on aratio between the distance between the markers measured in the markerdistance measuring step and the maximum width of the display surface inthe second scale, as well as based on the predetermined distanceobtained in the distance obtaining step.

The fifth invention is the same as the above described fourth inventionother than the calculation in the display surface width calculationstep. In the display surface width calculation step, the maximum widthof the display surface (display width D) in the first scale iscalculated based on the ratio of the distance between the two markers(sensor-corresponding width W) in the second scale and the maximum widthof the display surface (the maximum display width M) in the secondscale, and based on the predetermined distance between the two markers(the sensor width S) in the first scale. Therefore, the adjustment canbe readily performed even if the user does not know the maximum width ofthe display surface of the display apparatus.

A sixth invention provides a storage medium having stored thereon aprogram for adjusting a pointing device according to the fourthinvention. In the current distance calculating step, the currentdistance is calculated using the distance between the two markers on theimaged image obtained by the imaging means.

According to the sixth invention, in the current distance calculatingstep, the current distance (Lc) is calculated using the distance betweenthe two markers (Si) on the imaged image. Therefore, the user may easilyadjust by changing the distance from the display apparatus or theimaging objective while holding the input apparatus and imaging theimaging objective.

A seventh invention provides a storage medium having stored thereon aprogram for adjusting a pointing device according to the fifthinvention. In the current distance calculating step, the currentdistance is calculated using the distance between the two markers on theimaged image obtained by the imaging means. The effect of the seventhinvention is the same as that of the sixth invention.

A eighth invention provides a storage medium having stored thereon aprogram for adjusting a pointing device according to the firstinvention. The display surface width obtaining step includes a displaysurface width inputting step of obtaining the maximum width, inputted bythe user, of the display surface of the display apparatus.

According to the eighth invention, in the display surface widthinputting step (S3′ and S105), the maximum width of the display surface(display width D) is obtained, for example, based on the data from theinput apparatus according to an operation, by the user, of an operationmeans (72) provided for the input apparatus. Therefore, it is possiblefor the user to input the display width if the user knows the maximumwidth of the display surface of the display apparatus.

A ninth invention provides a pointing device for obtaining input datafrom an input apparatus provided with an imaging means for imaging aknown imaging objective, the input data being one of data of an imagedimage obtained by the imaging means and data obtained by performing apredetermined operation on the obtained data of the imaged image, andperforming information processing using the input data to output to adisplay apparatus. The pointing device includes: a display surface widthobtaining means, an appropriate distance determining means, a currentdistance calculating means, and a notifying means. The display surfacewidth obtaining means obtains a maximum width of a display surface ofthe display apparatus. The appropriate distance determining meansdetermines an appropriate distance between the input apparatus and thedisplay apparatus according to the maximum width of the display surfaceobtained by the display surface width obtaining means. The currentdistance calculating means calculates a current distance from the inputapparatus to the imaging objective based on the input data. Thenotifying means makes a notification according to relation between theappropriate distance and the current distance.

A tenth invention provides a pointing device according to the ninthinvention. The notifying means makes the notification by one ofoutputting image display, outputting sound such as a voice sound and asound effect, and having the input apparatus vibrate, according to adifference between the current distance and the appropriate distance.

An eleventh invention provides a pointing device according to the ninthinvention. The notifying means makes the notification by displaying, onthe display apparatus, a reference image having a predetermined size andan adjustment image varying in size in conjunction with the currentdistance and becoming the same size as the reference image when thecurrent distance is equal to the appropriate distance.

A twelfth invention provides a pointing device according to the ninthinvention. The known imaging objective is constituted by two markerspositioned with a predetermined distance therebetween measured in afirst scale, the two markers being positioned in a vicinity of apredetermined side of the display surface of the display apparatus sothat a positioning direction of the two markers is in parallel to thepredetermined side. The pointing device further comprises a distanceobtaining means for obtaining the predetermined distance either inputtedby a user or previously stored. The display surface width obtainingmeans includes: a marker distance measuring means for measuring adistance between the two markers in a second scale by varying aninterval in the positioning direction between the two markers in animage displayed on the display surface, according to the input by theuser; and a display surface width calculating means for calculating themaximum width of the display surface in the first scale based on a ratiobetween the predetermined distance obtained by the distance obtainingmeans and the distance between the markers measured by the markerdistance measuring means, as well as based on the maximum width of thedisplay surface in the second scale.

A thirteenth invention provides a pointing device according to the ninthinvention. The known imaging objective is constituted by two markerspositioned with a predetermined distance therebetween measured in afirst scale, the two markers being positioned in a vicinity of apredetermined side of the display surface of the display apparatus sothat a positioning direction of the two markers is in parallel to thepredetermined side. The pointing device further comprises a distanceobtaining means for obtaining the predetermined distance either inputtedby a user or previously stored. The display surface width obtainingmeans includes: a marker distance measuring means for measuring adistance between the two markers in a second scale by varying aninterval in the positioning direction between the two markers in animage displayed on the display surface, according to the input by theuser; and a display surface width calculating means for calculating themaximum width of the display surface in the first scale based on a ratiobetween the distance between the markers measured by the marker distancemeasuring means and the maximum width of the display surface in thesecond scale, as well as based on the predetermined distance obtained bythe distance obtaining means.

A fourteenth invention provides a pointing device according to thetwelfth invention. The current distance calculating means calculates thecurrent distance using the distance between the two markers on theimaged image obtained by the imaging means.

A fifteenth invention provides a pointing device according to thethirteenth invention. The current distance calculating means calculatesthe current distance using the distance between the two markers on theimaged image obtained by the imaging means.

A sixteenth invention provides a pointing device according to the ninthinvention. The display surface width obtaining means includes a displaysurface width inputting means for obtaining the maximum width, inputtedby the user, of the display surface of the display apparatus.

The ninth to sixteenth inventions relate to pointing devicescorresponding to the above described storage medium of the first toeighth inventions, and have the similar effects therewith.

According to example embodiments of the present invention, the relationbetween the current distance between the imaging objective and the inputapparatus and the appropriate distance may be notified, and therefore,it is possible to set the most appropriate setting for the useenvironment. The user may operate with an appropriate distance in whichthe position or the direction pointed by the user matches the positionor the direction recognized on the display screen.

The above described objects and other objects, features, aspects andadvantages of the present invention will become more apparent from thefollowing detailed description of the present invention when taken inconjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an illustrative view showing an appearance of a gaming systemaccording to one embodiment of the present invention;

FIG. 2 is a block diagram showing an electrical configuration of thegaming system shown in FIG. 1;

FIG. 3 is an illustrative view showing an appearance of a controllershown in FIG. 1, where FIG. 3(A) is an oblique perspective view fromrear top, and FIG. 3(B) is an oblique perspective view from frontbottom;

FIG. 4 is a block diagram showing an electrical configuration of acontroller shown in FIG. 1;

FIG. 5 is an illustrative view for explaining a state where a game isplayed using the controller shown in FIG. 1;

FIG. 6 is an illustrative view for explaining a viewing angle of markersand the controller shown in FIG. 1;

FIG. 7 is an illustrative view of one example of imaged images includingan object image;

FIG. 8 is an illustrative view for explaining an input of asensor-corresponding width for calculating a display width;

FIG. 9 is an illustrative view of positional relation, among thecontroller, a display screen, and the markers, for calculating anappropriate distance, where FIG. 9(A) shows the positional relation whena left end of the display screen is ideally pointed, and FIG. 9(B) showsthe positional relation for limit of recognition when waving thecontroller to the left;

FIG. 10 is an illustrative view for explaining a method of calculating acurrent distance from the two markers to the controller;

FIG. 11 is an illustrative view showing one example of images fornotifying the relation between the current distance and the appropriatedistance, where FIG. 11(A) shows a case where the current distance islonger than the appropriate distance, and FIG. 11(B) shows a case wherethe current distance is shorter than the appropriate distance;

FIG. 12 is an illustrative view showing one example of a memory map of amain memory;

FIG. 13 is an illustrative view showing table data of the appropriatedistance;

FIG. 14 is a flowchart showing one example of an operation of anadjustment process of a video game apparatus;

FIG. 15 is a flowchart showing one example of an operation of a displaywidth calculation process shown in FIG. 14;

FIG. 16 is a flowchart showing one example of an operation of a currentdistance calculation process shown in FIG. 14;

FIG. 17 is a flowchart showing an operation of an adjustment process ofanother embodiment;

FIG. 18 is an illustrative view showing one example of a display sizeinput screen displayed in another embodiment;

FIG. 19 is an illustrative view showing one example of an appropriatedistance table of the embodiment shown in FIG. 18; and

FIG. 20 is a flowchart showing an operation of an adjustment process ofthe embodiment shown in FIG. 18.

DETAILED DESCRIPTION OF THE EXAMPLE EMBODIMENTS

Referring to FIG. 1, a pointing device is formed with a configuration ofa game system of one embodiment of the present invention. The gamesystem 10 includes a video game apparatus 12 and a controller 22. Thevideo game apparatus 12 and the controllers 22 are connected by radio.For example, wireless communication is executed according to Bluetooth(registered trademark) standard, and may be executed by other standards,such as infrared rays and wireless LAN.

The video game apparatus 12 includes a roughly rectangularparallelepiped housing 14, and the housing 14 is provided with a diskslot 16 on a front surface. An optical disk 18 (see FIG. 2) as oneexample of an information storage medium storing game program and datais inserted into the disk slot 16 to be loaded in a disk drive 68 (seeFIG. 2) within the housing 14.

On the front surface of the housing 14 of the video game apparatus 12and near the disk slot 16 is provided a memory card slot cover 28.Inside the memory card slot cover 28, a memory card slot (notillustrated) is provided into which an external memory card(hereinafter, simply referred to as “memory card”) 30 (see FIG. 2) isinserted. The memory card 30 is employed for loading the game program,etc. read from the optical disk 18 to temporarily store it, storing(saving) game data (result data or proceeding data of the game) of thegame played by means of the game system 10, and so forth. It should benoted that storing the game data described above may be performed, inplace of the memory card 30, on an internal memory by providing theinternal memory, such as a flash memory, etc. inside the video gameapparatus 12.

The video game apparatus 12 has an AV cable connector (not illustrated)on the rear surface of the housing 14, and by utilizing the connector, amonitor 34 is connected to the video game apparatus 12 via an AV cable32 a. The monitor 34 is typically a color television receiver, and theAV cable 32 a inputs a video signal from the video game apparatus 12 toa video input terminal of the color television, and inputs a soundsignal to a sound input terminal. Accordingly, a game image of athree-dimensional (3D) video game, for example, is displayed on thescreen of the color television (monitor) 34, and stereo game sound, suchas a game music, a sound effect, etc. is output from right and leftspeakers 34 a.

Additionally, around the monitor 34 (upper side in this embodiment), asensor bar 34 b is provided, and the sensor bar 34 b is provided withtwo LED modules (hereinafter referred to as “marker”) 340 m and 340 nwith a predetermined distance therebetween. Each of the markers 340 mand 340 n is an infrared LED, and outputs infrared light toward thefront of the monitor 34. A cable 32 b extending from the sensor bar 34 bis connected to a connector (not illustrated) on a rear surface of thevideo game apparatus 12, and a power is supplied to the markers 340 mand 340 n from the video game apparatus 12.

Furthermore, the power of the video game apparatus 12 is applied bymeans of a general AC adapter (not illustrated). The AC adapter isinserted into a standard wall socket for home use, and the video gameapparatus 12 transforms the house current (commercial power supply) to alow DC voltage signal suitable for driving. In another embodiment, abattery may be utilized as a power supply.

In the game system 10, a user or a player turns the power of the videogame apparatus 12 on for playing the game (or other applications). Then,the user selects an appropriate optical disk 18 storing a video game (orother applications the player wants to play), and loads the optical disk18 on the disk drive 68 of the video game apparatus 12 through the diskslot 16. In response thereto, the video game apparatus 12 starts toexecute a video game or other applications on the basis of the softwarestored in the optical disk 18. The user operates the controller 22 inorder to apply an input to the video game apparatus 12.

FIG. 2 is a block diagram showing an electric configuration of the videogame system 10 in FIG. 1 embodiment. A CPU 36 is provided in the videogame apparatus 12. The CPU 36 is a computer which executes an overallcontrol of the video game apparatus 12. The CPU 36 functions as a gameprocessor, and is connected with a memory controller 38 via a bus. Thememory controller 38 mainly controls writing and reading of a mainmemory 40 connected via the bus under the control of the CPU 36. Thememory controller 38 is connected with a GPU (Graphics Processing Unit)42.

The GPU 42 forms a part of a rendering means, and is constituted by asingle chip ASIC, for example, receives a graphics command (renderingcommand) from the CPU 36 via the memory controller 38, and by followingthe command thus received, generates a 3D game image by a geometry unit44 and a rendering unit 46. Namely, the geometry unit 44 performsarithmetic processing of rotation, movement, and deformation, etc, ofeach kind of object of three dimensional coordinate system (formed of aplurality of polygons, and the polygon refers to a polygonal planedefined by at least three vertexes coordinates.) The rendering unit 46performs image generation processing such as attaching a texture(texture image) to each polygon of each kind of object, and so forth.Accordingly, the 3D image data to be displayed on the game screen isgenerated by the GPU 42, and the image data thus generated is stored ina frame buffer 48.

Note that necessary data (primitive or polygon and texture, etc) inperforming the graphics command by the GPU 42 is obtained from the mainmemory 40 by the GPU 42 via the memory controller 38.

The frame buffer 48 is a memory for drawing (accumulating) the imagedata of one frame of a raster scan monitor 34, for example, and isoverwritten for every one frame by the GPU 42. Specifically, the framebuffer 48 sequentially stores chromatic information of an image for eachone pixel. Here, the chromatic information refers to data on R, G, B, A,and for example, corresponds to R (red) data of 8 bits, G (green) dataof 8 bits, B (blue) data of 8 bits, and A (alpha) data of 8 bits. Notethat A data is the data on a mask (mat image). The 3D image of the gameimage is displayed on the screen of the monitor 34 by reading the dataof the frame buffer 48 via the memory controller 38 by a video I/F 58 aswill be described later.

In addition, a Z buffer 50 has a storage capacity corresponding to thenumber of pixels corresponding to the frame buffer 48 X the number ofbits of depth data per one pixel, and stores depth information or depthdata (Z value) of dots corresponding to each storage location of theframe buffer 48.

Both of the frame buffer 48 and the Z buffer 50 may be constituted byusing one portion of the main memory 40, and also these buffers may beprovided inside the GPU 42.

In addition, the memory controller 38 is connected to a RAM for audio(referred to as “ARAM” hereafter) 54, via a DSP (Digital SignalProcessor) 52. Accordingly, the memory controller 38 controls writingand/or reading of the ARAM 54 as a sub-memory as well as that of themain memory 40.

The DSP 52 works as a sound processor, and generates audio datacorresponding to sound, voice, sound effect or music necessary for thegame, by using sound data (not shown) stored in the main memory 40 andby using sound wave (tone) data (not shown) written in the ARAM 54.

The memory controller 38 is further connected to each interface (I/F)56, 58, 60, 62, and 64 by the bus. The controller I/F 56 is an interfacefor the controller 22 connected to the video game apparatus 12 via aBluetooth communication unit 66. More specifically, the Bluetoothcommunication unit 66 receives input data sent from the controller 22,and the controller I/F 56 applies the input data to the CPU 36 throughthe memory controller 38. It should be noted that in this embodiment,the input data includes at least any one of operation data, accelerationdata, and marker coordinate data described later. Also, the Bluetoothcommunication unit 66 receives data produced by the CPU 36 via the mainmemory 40 and the controller I/F 56, and transmits the data to thecontroller 22 to be transmitted. The CPU 36 produces sound outputinstruction data including audio data generated at the DSP 52 whenoutputting sound from a speaker 104 of the controller 22 (see FIG. 4),and has the Bluetooth communication unit 66 transmit the produced datato the controller 22. In addition, the CPU 36 produces vibration outputinstruction data including vibration control data when vibrating avibrator 106 of the controller 22 (see FIG. 4), and has the produceddata be transmitted from the Bluetooth communication unit 66. Moreover,the CPU 36 produces lighting instruction data including lighting controldata when lighting the LED 76 of the controller 22 (see FIG. 3 and FIG.4), and has the produced data be transmitted from the Bluetoothcommunication unit 66.

The video I/F 58 accesses the frame buffer 48, reads the image datagenerated by the GPU 42, and applies an image signal or the image data(digital RGBA pixel value) to the monitor 34 via the AV cable 32 a (FIG.1).

The external memory I/F 60 associates the memory card 30 inserted intothe front face of the video game apparatus 12 with the memory controller38. Whereby, the CPU 36 can write the data into the memory card 30, orcan read out the data from the memory card 30 via the memory controller38. The audio I/F 62 receives audio data given from the DSP 52 throughthe memory controller 38 or audio stream read from the optical disk 18,and gives an audio signal (sound signal) corresponding thereto to aspeaker 34 a of the monitor 34.

Further, the disk I/F 64 connects the disk drive 68 to the memorycontroller 38, and therefore the CPU 36 controls the disk drive 68. Bythis disk drive 68, program and data read out from the optical disk 18are written into the main memory 40 under the control of the CPU 36.

Additionally, in FIG. 2, for simplicity, the sensor bar 34 b and thepower supply circuit are omitted.

FIG. 3 shows one example of an external appearance of the controller 22.FIG. 3 (A) is a perspective view viewing the controller 22 from aboverear, and FIG. 3 (B) is a perspective view viewing the controller 22from below front.

The controller 22 has a housing 70 formed by plastic molding, forexample. The housing 70 is formed into an approximately rectangularparallelepiped shape regarding a back and forth direction (Z-axisdirection shown in FIG. 3) as a longitudinal direction, and has a sizesmall enough to be held by one hand of a child and an adult. As oneexample, the housing 70 has a length or a width approximately the sameas that of the palm of the person. A player can perform a game operationby means of the controller 22, that is, by pushing buttons provided onit and by changing a position and a direction of the controller 22itself.

The housing 70 is provided with a plurality of operation buttons. Thatis, on the top surface of the housing 70, a cross key 72 a, an X button72 b, a Y button 72 c, an A button 72 d, a select switch 72 e, a menuswitch 72 f, and a start switch 72 g are provided. Meanwhile, on thebottom surface of the housing 70, a concave portion is formed, and onthe reward inclined surface of the concave portion, a B button 72 h isprovided. Each of the buttons (switches) 72 a-72 h is assigned anappropriate function according to a game program to be executed by thegame apparatus 12. Furthermore, the housing 70 has a power switch 72 ifor turning on/off the power of the main body of the game apparatus 12on a top surface from a remote place. The respective buttons (switches)provided on the controller 22 may inclusively be indicated with the useof the reference numeral 72.

At the back surface of the housing 70, an external expansion connector74 is provided. At the back end of the top surface of the housing 70, aplurality of LEDs 76 is provided, and the plurality of LEDs 76 shows acontroller number (identification number of the controller) of thecontroller 22. The game apparatus 12 can be connected with a maximumfour controllers 22, for example. If a plurality of controllers 22 isconnected to the game apparatus 12, a controller number is applied tothe respective controllers 22 in the order connected, for example. EachLED 76 corresponds to a controller number, and the LED 76 correspondingto the controller number lights up.

Furthermore, inside the housing 70 of the controller 22, an accelerationsensor 78 (FIG. 4) is provided. As an acceleration sensor 78,acceleration sensors of an electrostatic capacity type can typically beutilized. The acceleration sensor 78 detects accelerations of a linearcomponent for each sensing axis and gravitational acceleration out ofthe accelerations applied to a detection portion of the accelerationsensor. The acceleration sensor 78 detects accelerations of at least twoaxes which are orthogonal with each other. For the two-axis orthree-axis acceleration sensor, for example, the acceleration applied tothe detection portion of the acceleration sensor is detected asaccelerations of linear components along the respective axes. Morespecifically, in this embodiment, a three-axis acceleration sensor isapplied to detect the respective accelerations in directions of threeaxes of a up and down direction (Y-axial direction shown in FIG. 3), aright and left direction (X-axial direction shown in FIG. 3), and aforward and rearward direction (Z-axial direction shown in FIG. 3) ofthe controller 22. The acceleration detected for each axis of theacceleration sensor 78 is subjected to a predetermined arithmeticprocess, to thereby calculate a slope and a rotation of the controller22. For example, gravitational acceleration is constantly applied evenin a state that the acceleration sensor 78 stands still, andacceleration corresponding to a slope of each axis with respect to thegravitational acceleration is detected for each axis. More specifically,when the acceleration sensor 78 stands still in a horizontal state (thatis, X and Z-axis directions are a horizontal direction, and a Y-axisdirection is a vertical direction), the gravitational acceleration of 1G is applied to the Y-axis of the acceleration sensor 78, and thegravitational accelerations to the other X and Z-axes are 0. When theacceleration sensor 78 is sloped from the horizontal state, thegravitational acceleration is distributed into each axis of theacceleration sensor 78 depending on angles formed by each axis directionof the acceleration sensor 78 and the gravitational direction, and atthis time, an acceleration value of each axis of the acceleration sensor78 is detected. An arithmetic operation is performed on the accelerationvalue for each axis, to thereby calculate the orientation of theacceleration sensor 78 with respect to the direction of thegravitational acceleration.

It should be noted that as an acceleration sensor 78, two-axisacceleration sensors may be utilized for detecting any two of thedirections of the accelerations out of the up and down direction, theright and left direction and the back and forth direction according tothe shape of the housing 70, the limitation on how to hold thecontroller 22, or the like.

In addition, the controller 22 has an imaged information arithmeticsection 80 (see FIG. 4), and as shown in FIG. 3 (B), on the front endsurface of the housing 70, light incident opening 82 of the imagedinformation arithmetic section 80 is provided.

Furthermore, a plurality of holes 102 is formed between the menu switch72 f and the X button 72 b on the top surface of the housing 70. Aspeaker 104 (see FIG. 4) is provided inside the housing 70 at theposition corresponding to these holes 102, and therefore it is possibleto provide the player with a sound form the controller 22 at hand too.

In addition, a vibrator 106 (see FIG. 4) is provided inside the housing70. The vibrator 106 may be a vibration motor or a solenoid, forexample. The vibrator 106 produces a vibration in the controller 22, andtherefore it is possible to transmit the vibration to the hand of theplayer holding it.

Furthermore, the controller 22 is powered by a battery (not illustrated)detachably incorporated in the housing 70. Additionally, the shapes ofthe controller 22 shown in FIG. 3 and the shape, the number and thesetting position of the buttons (switches, stick, or the like) aremerely one example, and can be changed to other shapes, numbers andsetting positions, etc. as needed.

FIG. 4 is a block diagram showing the electric configuration of thecontroller 22. Referring to FIG. 4, the controller 22 includes

The controller 22 incorporates a communication unit 84, and thecommunication unit 84 is connected with the operating portion 72, theacceleration sensor 78, the imaged information arithmetic section 80,the speaker 104, the vibrator 106 and the LEDs 76. It should be notedthat although omitted in FIG. 4, the communication unit 84 is alsoconnected with above-described connector 74 and the power supply circuitnot shown, etc.

The communication unit 84 includes a microcomputer (micon) 86, a memory88, a radio module 90 and an antenna 92. The micon 86 transmits theobtained data to the game apparatus 12 and receives data from the gameapparatus 12 by controlling the radio module 90 while using the memory88 as a memory area (working area and buffer area) in processing.

The operating portion 72 indicates the above-described operation buttonsor operation switches 72 a-72 i. When the operating portion 72 isoperated, a manipulate signal is applied to the communication unit 84.

The data indicative of acceleration of each axis detected by theacceleration sensor 78 is output to the communication unit 84. Theacceleration sensor 78 has in the order of a maximum sampling period of200 frames per second, for example.

The data obtained by the imaged information arithmetic section 80 isalso output to the communication unit 84. The imaged informationarithmetic section 80 is made up of an infrared filter 94, a lens 96, animaging element 98, and an image processing circuit 100. The infraredfilter 94 passes only infrared rays from the light incident opening 82in front of the controller 22. As described above, the markers 340 m and340 n placed near (around) the display screen of the monitor 34 areinfrared LEDs for outputting infrared lights forward the monitor 34.Accordingly, by providing the infrared filter 94, it is possible toimage the image of the markers 340 m and 340 n more accurately. The lens96 condenses the infrared rays passing thorough the infrared filter 94to emit them to the imaging element 98. The imaging element 98 is asolid-state image sensing device, such as a CMOS sensor and a CCD, forexample, and images the infrared rays condensed by the lens 96.Accordingly, the imaging element 98 images only the infrared rayspassing through the infrared filter 94 to generate image data.Hereafter, the image imaged by the imaging element 80 c is called an“imaged image”. The image data generated by the imaging element 98 isprocessed by the image processing circuit 100. The image processingcircuit 100 calculates a position of an imaging objective (markers 340 mand 340 n) within the imaged image, and outputs marker coordinate dataincluding each coordinate value indicative of the position to the micon86 at predetermined time intervals (one frame, for example). It shouldbe noted that a description of processing of the image processingcircuit 100 is made later.

The data output from the operating portion 72, the acceleration sensor78, and the imaged information arithmetic section 80 to the micon 86 istemporarily stored in the memory 88. The radio transmission from thecommunication unit 84 to the Bluetooth communication unit 66 of the gameapparatus 12 is performed every predetermined cycle. The game processingis generally performed by regarding 1/60 seconds as a unit, andtherefore, it is necessary to perform the transmission from thecontroller 22 at a cycle equal to or shorter than it. The micon 86outputs data including the operation data of the operating portion 72,the acceleration data of the acceleration sensor 78, and the markercoordinate data of the imaged information arithmetic section 80, etc.stored in the memory 88 to the radio module 90 as controller data (inputdata), when transmission timing to the game apparatus 12 has come. Theradio module 90 modulates a carrier of a predetermined frequency by theinput data, and emits its weak radio wave signal from the antenna 92 byusing a short-range wireless communication technique, such as Bluetooth.Namely, the input data is modulated to the weak radio wave signal by theradio module 90 and transmitted from the controller 22. The weak radiowave signal is received by a Bluetooth communication unit 66 of the gameapparatus 12. The weak radio wave thus received is subjected todemodulating and decoding processing, thus making it possible for thegame apparatus 12 to obtain the input data. The CPU 36 of the gameapparatus 12 performs the game processing on the basis of the input dataobtained from the controller 22.

Moreover, the communication unit 84 receives a weak radio signal fromthe game apparatus 12 via the antenna 92, demodulates and decodes thereceived weak radio signal by the radio module 90, and obtains the datatransmitted from the game apparatus 12. When the received data is datainstructing sound output, the micon 86 outputs the audio data includedin the received data to the speaker 104 via a D/A converter and anamplifier not shown in the drawing, and has the speaker 104 outputs thesound. Further, when the received data is data instructing vibrationoutput, the micon 86 controls vibration of the vibrator 106 based on thedata. In addition, when the received data is data instructing lightingof the LED 76, the micon 86 controls lighting of the LED 76 based on thedata.

FIG. 5 is an illustrative view summarizing a state when a player plays agame by utilizing a controller 22. As shown in FIG. 5, when playing thegame by means of the controller 22 in the video game system 10, theplayer holds the controller 22 with one hand. Strictly speaking, theplayer holds the controller 22 in a state that the front end surface(the side of the light incident opening 82 of the light imaged by theimaged information arithmetic section 80) of the controller 22 isoriented to the markers 340 m and 340 n. It should be noted that as canbe understood from FIG. 1, the markers 340 m and 340 n are placed in avicinity of a predetermined side (upper side or lower side) of thescreen of the monitor 34 and in parallel with the predetermined side. Inthis state, the player performs a game operation by moving thecontroller 22, that is, changing a position on the screen indicated bythe controller 22, and changing a distance between the controller 22 andeach of the markers 340 m and 340 n.

FIG. 6 is a view showing viewing angles between the respective markers340 m and 340 n, and the controller 22. As shown in FIG. 6, each of themarkers 340 m and 340 n emits infrared ray within a range of a viewingangle θ1. Also, the imaging element 98 of the imaged informationarithmetic section 80 can receive incident light within the range of theviewing angle α taking the line of sight of the controller 22 (Z axialdirection shown in FIG. 3) as a center. For example, the viewing angleθ1 of each of the markers 340 m and 340 n is 34° (half-value angle)while the viewing angle α of the imaging element 98 is 42°. The playerholds the controller 22 such that the imaging element 98 is directed andpositioned so as to receive the infrared rays from the markers 340 m and340 n. More specifically, the player holds the controller 22 such thatat least one of the markers 340 m and 340 n exists in the viewing angleα of the imaging element 98, and the controller 22 exists in at leastone of the viewing angles θ1 of the marker 340 m or 340 n. In thisstate, the controller 22 can detect at least one of the markers 340 mand 340 n. The player can perform a game operation by changing theposition and the orientation of the controller 22 in the rangesatisfying the state. It should be noted that when only one of themarkers 340 m and 340 n is detected, instructed position of thecontroller 22 can be calculated, by utilizing the data detectedimmediately before in which the two markers 340 m and 340 n aredetected, and setting temporary marker coordinate in place of the othermarker that is not detected, for example.

If the position and the orientation of the controller 22 are out of therange, the game operation based on the position and the orientation ofthe controller 22 cannot be performed. Hereafter, the above-describedrange is called an “operable range.”

If the controller 22 is held within the operable range, an image of eachof the markers 340 m and 340 n is imaged by the imaged informationarithmetic section 80. That is, the imaged image obtained by the imagingelement 98 includes an image (object image) of each of the markers 340 mand 340 n as an imaging objective. FIG. 7 is a view showing one exampleof the imaged image including an object image. The image processingcircuit 100 calculates coordinates (marker coordinates) indicative ofthe position of each of the markers 340 m and 340 n in the imaged imageby utilizing the image data of the imaged image including the objectimage 340 m′ and 340 n′.

Since the object images 340 m′ and 340 n′ appear as a high-intensitypart in the image data of the imaged image, the image processing circuit100 first detects the high-intensity part as a candidate of the objectimage. Next, the image processing circuit 100 determines whether or notthe high-intensity part is an object image on the basis of the size ofthe detected high-intensity part. The imaged image may include imagesother than the object image due to sunlight through a window and lightof a fluorescent lamp in the room as well as the images 340 m′ and 340n′ of the two markers 340 m and 340 n. The determination processingwhether or not the high-intensity part is an object image is executedfor discriminating the images 340 m′ and 340 n′ of the two markers 340 mand 340 n from the images other than them, and accurately detecting theobject image. The imaging objective 340 m and 340 n need to be known fordiscriminating the object images 340 m′ and 340 n′ from the other imagesin the imaged image. In this embodiment, the size of the imagingobjective is determined in advance, and therefore, it is possible toestimate the size of the marker images 340 m′ and 340 n′. Accordingly,it is possible to execute a determination of the marker images 340 m′and 340 n′ based on the size of the high-intensity part. Morespecifically, in the determination process, it is determined whether ornot the detected high-intensity part is within the size of the presetpredetermined range. Then, if the high-intensity part is within the sizeof the predetermined range, it is determined that the high-intensitypart represents the object image. On the contrary, if the high-intensitypart is not within the size of the predetermined range, it is determinedthat the high-intensity part represents the images other than the objectimage.

In addition, as to the high-intensity part which is determined torepresent the object image as a result of the above-describeddetermination processing, the image processing circuit 100 calculatesthe position of the high-intensity part. More specifically, thebarycenter position of the high-intensity part is calculated. Here, thecoordinates of the barycenter position is called a “marker coordinate”.Also, the barycenter position can be calculated with more detailed scalethan the resolution of the imaging element 98. Now, the resolution ofthe imaged image imaged by the imaging element 98 shall be 126×96, andthe barycenter position shall be calculated with the scale of 1024×768.That is, the marker coordinate is represented by the integer from (0, 0)to (1024, 768).

Additionally, the position in the imaged image shall be represented by acoordinate system (XY coordinate system of the imaged image) taking theupper left of the imaged image as an origin point, the downwarddirection as an Y-axis positive direction, and the right direction as anX-axis positive direction.

Also, if the object image is properly detected, two high-intensity partsare determined as an object image by the determination process, andtherefore, two marker coordinates are calculated. The image processingcircuit 100 outputs data indicative of the calculated two markercoordinates. The output data (marker coordinate data) is included in theinput data by the micon 86 as described above, and transmitted to thevideo game apparatus 12.

The video game apparatus 12 (CPU 36) detects the marker coordinate datafrom the received input data to thereby calculate an instructed position(instructed coordinate) by the controller 22 on the screen of themonitor 34 and a distance from the controller 22 to each of the markers340 m and 340 n on the basis of the marker coordinate data. Morespecifically, the position of the mid point of the two markercoordinates is adopted (calculated) as a position to which thecontroller 22 faces, that is, an instructed position. It should be notedthat when the controller 22 points to the left end of the screen of themonitor 34, the object images are detected in right side in the imagedimage, and when the controller 22 points to the lower end of the screen,the object images are detected in the upper side in the imaged image,for example. Therefore, when the coordinate of the instructed positionby the controller 22 is calculated from the marker coordinate, thecoordinate system is appropriately transformed from the coordinatesystem of the imaged image of FIG. 7.

In this embodiment, marker coordinates are detected by performing apredetermined calculation process with the controller 22 on imaged data,and then, the marker coordinate data is transmitted to the video gameapparatus 12. However, in another embodiment, the imaged data may betransmitted as input data from the controller 22 to the video gameapparatus 12, and the CPU 36 of the video game apparatus 12 may detectthe marker coordinates by performing the predetermined calculationprocess on the imaged data.

Further, a distance between the object images in the imaged image variesdepending on the distances between the controller 22 and the markers 340m and 340 n. Therefore, by calculating a distance between the two markercoordinates, the video game apparatus 12 can perceive the distancesbetween the controller 22 and the markers 340 m and 340 n.

As described above, this gaming system 10 is provided with a mechanismas a pointing device for instructing a position on a display surface ofthe monitor 34 with the controller 22. The video game apparatus 12performs information processing such as various gaming processes usingthe input data from the controller 22, and displays the results on themonitor 34 as a display apparatus. For example, the video game apparatus12 determines, based on the position pointed by the controller 22 and aposition of an object on the display screen, whether or not the objectis pointed, and performs a gaming process corresponding to the result.As an example, by directing a Z axis of the controller 22 toward atarget on the display screen, it is possible to execute a game in whichthe target may be aimed and shot.

However, a size of the display surface of the monitor 34 and a positionwhere the player stands vary depending on circumstances of each family,and accordingly, there is a risk that the position on the display screenat which the player intends to point does not match the positionrecognized by the gaming system 10. Therefore, in order to match theposition pointed by the player with the controller 22 and the positionrecognized on the display screen, this gaming system 10 is configured sothat the distance between the controller 22 and the display screen(markers 340 m and 340 n) can be adjusted so as to be an appropriatedistance.

The appropriate distance from the monitor 34 (markers 340 m and 340 n)varies depending on a width of the display surface of the monitor 34 andan interval between the markers 340 m and 340 n. Accordingly, in orderto calculate the appropriate distance, it is necessary to obtaininformation about the interval between the markers 340 m and 340 n andthe width of the display surface of the monitor 34. In this embodiment,the width of the sensor bar 34 b is previously determined, i.e., the twomarkers 340 m and 340 n are positioned with a predetermined distancetherebetween, and the width of the display surface of the monitor 34 isobtained using this width of the sensor bar 34 b.

As shown in FIG. 8, the two markers 340 m and 340 n in the sensor bar 34b are positioned in a vicinity of one of upper and lower sides of thedisplay surface of the monitor 34 so that a positioning direction of thetwo markers is parallel with the one of the sides. Further, the sensorbar 34 b is positioned in the center of the one of the sides so that acenter point of the two markers 340 m and 340 n matches a center of thedisplay surface in widthwise direction. Then, an image 200 with avariable width is displayed in the center of the display screen, and theplayer is requested to operate so as to change a width of the image 200on the display screen in a direction in which the two markers 340 m and340 n are positioned to match with the interval between the two markers340 m and 340 n. For example, the display of the image 200 is controlledso that pressing a right direction of the cross key 72 a of thecontroller 22 widens the width of the image 200 in widthwise directionby a predetermined length (dot), and pressing a left direction of thecross key 72 a narrows the width of the image 200 by a predeterminedlength. By operating this cross key 72 a so that the width of the image200 matches the width of the sensor bar 34 b and pressing an A button 72d, the width of the image 200 may be determined, and accordingly, theinterval between the two markers 340 m and 340 n may be measured basedon a unit (pixel) of the image. The width of the image 200 is referredto as a sensor-corresponding width. Specifically, thesensor-corresponding width is the width within the display screencorresponding to an actual width distance of the sensor bar 34 b, andindicates the distance in the image. Note that the width of the sensorbar 34 b, i.e., an actual distance between the markers 340 m and 340 nis referred to as a sensor width.

Because a maximum width of an image capable of being displayed on thedisplay screen (referred to as a maximum display width) is previouslydetermined, it is possible to calculate the width of the display screenof the monitor 34 (referred to as a display width) by obtaining thesensor-corresponding width. Scales or units for the display width, thesensor width, the sensor-corresponding width, and the maximum displaywidth are different. The display width and the sensor width areexpressed in a first scale (i.e. a unit for measuring a length in a realspace: m (meter)), and the sensor-corresponding width and the maximumdisplay width are expressed in a second scale (i.e. a unit for a digitalimage: dot (pixel)). The maximum display width is also the maximum widthof the display surface in the second scale.

When the display width is D (m), the sensor width is S (m), thesensor-corresponding width is W (dot), and the maximum display width isM (dot), the display width is calculated based on the following equation1.D=S×M/W  (1)

As described above, the display width is calculated based on a ratiobetween the sensor width and the sensor-corresponding width (S/W), andbased on the maximum display width. Alternatively, the display width iscalculated based on a ratio between the maximum display width and thesensor-corresponding width (M/W), and based on the sensor width.

As can be seen from FIG. 8, the display surface of the monitor 34 ofthis embodiment is a horizontally elongated rectangle, and the width ofthe display surface corresponds to the maximum width as is. As thedisplay width, the maximum width of the display surface is obtained.

Once the display width and the sensor width are determined, idealpositional relation for pointing to the display surface by thecontroller 22 is determined. Accordingly, it is possible to calculatethe appropriate distance between the controller 22 and the displaysurface based on the display width the sensor width.

FIG. 9 shows positional relation among the controller 22, the displaysurface, and the sensor bar 34 b for calculating the appropriatedistance. As one example, FIG. 9 illustrates a situation where the leftend of the display surface on which the sensor bar 34 b is positioned.

Specifically, FIG. 9(A) shows the positional relation between thecontroller 22 and the display surface when an ideal pointing to the leftend of the display screen is performed. FIG. 9A shows a situation of thepointing viewed from the top. Presumption is that the controller 22 ispositioned on a normal line of the display surface at the center.Further, in order to point the left end of the display screen, the Zaxis of the controller 22 is directed toward the left end of the displaysurface. This is a situation where the Z axis of the controller 22 isswung from the center of the display surface to the left by an angle θ.When the distance from the controller 22 to the display surface is L (m)and the display width is D, the angle θ in this pointing situation iscalculated by the following equation 2.θ=arctan(D/2L)  (2)

On the other hand, FIG. 9(B) shows positional relation as a limit of therecognition when the controller 22 is swung toward left, i.e., a limitfor imaging the two markers 340 m and 340 n of the sensor bar 34 b withthe imaging element 98 of the imaging information arithmetic section 80of the controller 22. A line defining a right end of a viewing angle aof the imaging element 98 matches the right end of the sensor bar 34 b.This is a situation where the Z axis of the controller 22 is swung fromthe center of the display surface to the left by an angle β. When thedistance from the controller 22 to the display surface (markers 340 mand 340 n) is L (m), the sensor width is S, and the viewing angle of theimaging element 98 is β, the angle β in this pointing situation iscalculated by the following equation 3.β=a/2−arctan(S/2L)  (3)

If the limit of the recognition of the pointed position by thecontroller 22 is when the controller 22 is directed toward the end ofthe display surface, the player can perform an ideal pointingcomfortably. Therefore, by calculating the distance L so that θ=β isestablished, the appropriate distance may be obtained.

As described above, the appropriate distance according to the width ofthe display surface of the monitor 34 and the distance between themarkers 340 m and 340 n can be grasped.

Next, a method for calculating the current distance from the markers 340m and 340 n to the controller 22 is explained by referring to FIG. 10.The current distance Lc is calculated using a distance Si between twomarker images 340 m′ and 340 n′ on the imaged image.

Specifically, in this embodiment, first, the distance Si between themarker images 340 m′ and 340 n′ on the imaged image is calculated. Asdescribed above, because coordinates of the images 340 m′ and 340 n′ onthe imaged image (i.e., marker coordinates) is detected, the distance Sican be calculated from the marker coordinates. The distance Si isexpressed in the second scale (dot) as the unit for measuring the lengthin the image.

Next, an imaging range width R is calculated. The imaging range width Rindicates the length in the real space corresponding to a width of arange actually imaged by the imaging element 98. Among the imaging rangewidth R, the sensor width S, an imaged image width Ri and the distanceSi between the marker images, relation of R:S=Ri:Si is established. Theimaged image width Ri (dot) is previously determined according to aspecification of the imaging element 98. Further, the sensor width S (m)indicating the distance between the markers 340 m and 340 n in the realspace is also fixed in this embodiment. Therefore, it is possible tocalculate the imaging range width R according to the following equation4.R=Ri×S/Si  (4)

The imaging range width R is expressed in the first scale (m) as a unitfor measuring the length in the real space.

Further, from the imaging range width R and the viewing angle a of theimaging element 98, the current distance Lc (m) can be calculatedaccording to the following equation 5.Lc=(R/2)/tan(a/2)  (5)

As described above, in the gaming system 10, the appropriate distanceand the current distance between the controller 22 and the monitor 34(markers 340 m and 340 n) can be obtained, it is possible to inform theplayer of magnitude relation of the current distance and the appropriatedistance. Therefore, the player can adjust the position of thecontroller 22, i.e. the distance from the player to the display screenso that the current distance becomes the appropriate distance. Themethod of calculating the distance from the marker to the controller 22described here is a mere example. Although not explained in detail, itis also possible to calculate the current distance using, for example, aso-called focal length from the lens 96 to an imaging plane of theimaging element 98, the size of the marker image on the imaging plane(the distance between the marker images in this embodiment), and thedistance between the real markers.

Notification of the relation between the appropriate distance and thecurrent distance is performed by, in this embodiment, outputting animage display corresponding to a difference between the appropriatedistance and the current distance.

FIG. 11 shows examples of the image for notifying of the relationbetween the current distance and the appropriate distance. FIG. 11(A)shows a case where the current distance is longer than the appropriatedistance, and FIG. 11(B) shows a case where the current distance isshorter than the appropriate distance. Displayed on the display screenare a reference image 202 (solid circle) indicating the appropriatedistance and an adjustment image 204 (dashed circle) indicating thecurrent distance. A size of the reference image 202 (diameter of thecircle) is fixed at a predetermined value, and, on the other hand, asize of the adjustment image 204 changes in accordance with the currentdistance. Specifically, when the current distance is longer than theappropriate distance, the adjustment image 204 becomes smaller than thereference image 202, and when the current distance is shorter than theappropriate distance, the adjustment image 204 becomes larger than thereference image 202. Further, when the current distance is equal to theappropriate distance, a size of the adjustment image 204 is as large asthat of the reference image 202. For example, the reference image 202and the adjustment image 204 of the reference size are provided, thenthe adjustment image 204 enlarged or reduced into the size obtained bymultiplying the ratio between the current distance Lc and theappropriate distance L (L/Lc) by the reference size is generated,thereby displaying the reference image 202 and the adjustment image 204.In this embodiment, the reference image 202 and the adjustment image 204of the predetermined reference size regardless of the value of theappropriate distance are prepared, and the size of the adjustment image204 is changed according to the ratio (L/Lc). However, in anotherembodiment, the reference size may be changed according to the value ofthe appropriate distance or the maximum width of the display surface.

Because the above notification corresponding to the relation between thecurrent distance and the appropriate distance is performed, the playermay easily adjust the position of the controller 22 to the position tobe the appropriate distance by moving the controller 22 back and forthwith respect to the monitor 34 with the Z axis direction of thecontroller 22 facing toward the sensor bar 34 b, and with the size ofthe adjustment image matching to the size of the reference image on thedisplay screen.

Note that, in another embodiment, it is also conceivable to display amessage instructing the player to move closer or away from the displayscreen, as an image indicating the relation between the current distanceand the appropriate distance, according to the difference between thecurrent distance and the appropriate distance. It is also conceivable todisplay a value indicating the difference between the current distanceand the appropriate distance.

Further, the notification method is not limited to the output of theimage display using the display apparatus 34, and it is possible toappropriately modify to a different outputs using a different outputapparatus. For example, it is also possible to notify by sound outputinstead of or along with the image display. Kinds, output patterns, orvolume of sound such as a voice sound or a sound effect may be changedaccording to the relation between the current distance and theappropriate distance, as in a case of the image as described above, sothat the player can know whether to move closer to or away from thedisplay screen. For example, in a case where the voice sound is used, avoice sound such as “move closer to the display screen” when the currentdistance is longer than the appropriate distance, a voice sound such as“move away form the display screen” when the current distance is shorterthan the appropriate distance, or a voice sound such as “appropriate”when the current distance is equal to the appropriate distance isoutputted from a speaker 34 a or the speaker 104 of the controller 22.Taking an error into account, the current distance may be determined tobe appropriate when the size of the difference between the currentdistance and the appropriate distance is not more than a predeterminedthreshold value. When the current distance is determined not to beappropriate, it is possible to output voice sound including, forexample, a value of a distance required to be the distance determined tobe appropriate as the appropriate distance. Furthermore, when using thesound effect, the distance to the display screen is notified by changinga kind, an output pattern, and a volume of the sound effect, accordingto the difference between the current distance and the appropriatedistance.

Moreover, it is also possible to notify by a vibration output by avibrator 104 of the controller 22 either independently or along with theimage display and/or sound output. Also in the vibration output,similarly as in the image and the sound, a pattern and a degree, etc. ofthe vibration are changed according to the relation between the currentdistance and the appropriate distance.

Further, it is also possible to notify by light output of the LED 76 ofthe controller 22 along with any of the above described notificationmethods or independently. Also in the light output, similarly as in theabove notification methods, a lighting pattern and brightness, etc. of aplurality of the LEDs 76 are changed according to the relation betweenthe current distance and the appropriate distance.

FIG. 12 shows one example of a memory map of the main memory 40. Themain memory 40 includes a program memory area 110 and a data memory area112. A part of a program and data is read from the optical disk 18entirely or partially and sequentially as needed, and stored in the mainmemory 40. Note that FIG. 12 shows only a part of the memory map, andother programs and data required for processing are also stored. Forexample, sound data for outputting voice sound, sound effect, and musicis read from the optical disk 18 and stored in the data memory area 112.Further, data of various predetermined numerical values such as themaximum display width M of the image displayed on the display surface,the viewing angle a of the imaging element 98, and the width Ri of theimaged image are also read from the optical disk 18 and stored in thedata memory area 112.

In a memory area 114 of the program memory area 110, a main program forexecuting a main routine for the adjustment process of this embodimentis stored.

In a memory area 116, a sensor width obtaining program is stored. Bythis program, the interval between the marker 340 m and the marker 340n, i.e., the sensor width S is obtained. In this embodiment, the sensorwidth S is previously determined, and therefore, the previously storedsensor width data is read from a predetermined area in the data memoryarea 112. Note that, in another embodiment, when the sensor width ismade variable, the sensor width inputted by the player can be obtainedbased on the input data with this program.

In a memory area 118, a display width obtaining program is stored. Bythis program, as shown in FIG. 8 in the above, the processes forobtaining the sensor-corresponding width W based on the input data andfor calculating the display width D are performed. Note that, in anotherembodiment, when the display width is inputted by the player, thedisplay width inputted by the player is obtained based on the inputdata.

In a memory area 120, an appropriate distance determination program isstored. By this program, the appropriate distance based on the displaywidth and the sensor width is determined. In this embodiment, asdescribed later, because appropriate distance table data is previouslystored, the appropriate distance corresponding to the display width isdetermined by referring to this table. Note that, in another embodiment,the appropriate distance may be calculated, based on the obtaineddisplay width D, the obtained sensor width S, and the previously storedviewing angle a of the imaging element 98, according to the equation 2and the equation 3 as described above when these equations are equal,without preparing the appropriate distance table data.

In memory area 122, a current distance calculation program is stored. Bythis program, the current distance between the controller 22 and themarkers 340 m and 340 n is calculated. The current distance, asdescribed referring to FIG. 10 in the above, is calculated using thedistance Si between the marker images 340 m′ and 340 n′ on the imagedimage by the imaging element 98. Specifically, the current distance Lcis calculated according to the above equation 5 based on the actualimaging range width R by the imaging element 98 and the viewing angle aof the imaging element 98. The imaging range width R is calculatedaccording to the above equation 4 based on the width Ri of the imagedimage, the actual distance between the markers 340 m and 340 n (thesensor width S), and the distance Si between the marker images.

In a memory area 124, a notification program is stored. By this program,the notification according to the relation between the current distanceand the appropriate distance is performed. In this embodiment, as shownin FIG. 11, the CPU 36 produces an image including the reference image202 of a predetermined size and the adjustment image 204 varying in sizeaccording to the difference between the current distance and theappropriate distance using the GPU 42, and displays the image on themonitor 34 via such as the video I/F 58. Further, when performing thenotification with sound, the CPU 36 uses the DSP 52 to produce audiodata for outputting the sound such as the voice sound and the soundeffect according to the difference between the current distance and theappropriate distance. When outputting from the speaker 34 a of themonitor 34, the CPU 36 outputs the sound based on the audio data fromthe speaker 34 a via the audio I/F 62 and such. Further, when outputtingfrom the speaker 104 of the controller 22, the CPU 36 transmits thesound output instruction data including the audio data to the controller22 via the Bluetooth communication unit 66 and such. Moreover, whennotifying by vibration, the CPU 36 produces the vibration control datafor outputting vibration according to the difference between the currentdistance and the appropriate distance, and transmits the vibrationoutput instruction data including the data to the controller 22. Inaddition, when the notification is performed by light output of the LED76, the lighting control data is produced in order to obtain a lightingstate according to the difference between the current distance and theappropriate distance, and transmits the lighting instruction dataincluding the data to the controller 22.

A memory area 126 of the data memory area 112 is an input data buffer,and controller data received every predetermined time (one frame) isstored. In a memory area 128, the sensor width data indicating thedistance between the two markers 340 m and 340 n of the sensor bar 34 bis stored. In this embodiment, the sensor bar 34 b in which thepositions of the two markers 340 m and 340 n are fixed is prepared, thesensor width S is fixed at the predetermined distance. In other words,the sensor width data is previously stored in the optical disk 18, andthe sensor width data is read and stored in the memory area 128.

In a memory area 130, image data for producing an image to be displayedon the monitor 34 is stored. For example, image data for displaying theimage 200 as shown in FIG. 8 for obtaining the sensor-correspondingwidth, the reference image 202 and the adjustment image 204 as shown inFIG. 11 (reference size), and various display screens and such isstored.

In a memory area 132, the current distance calculated by the abovecurrent distance calculation program is stored. In the memory area 134,scale factor data for enlarging/reducing the adjustment image 204 of thereference size is stored. This scale factor is, as described above, aratio between the appropriate distance L and the current distance Lc.

In a memory area 136, the appropriate distance table data is stored. Asshown in FIG. 13, the appropriate distance is stored in association withthe display width in the appropriate distance table data. The size ofthe display surface of the monitor 34 available on market is determined.Further, in this embodiment, the sensor width S is fixed. Therefore,according to the predetermined equation provided that θ in the aboveequation 2 and β in the equation 3 are equal, appropriate distance L1,L2 . . . Ln corresponding to display widths D1, D2, . . . Dn for all ofthe sizes of the monitor 34 are previously calculated and stored in theoptical disk 18 as table data. The above appropriate distancedetermination program in this embodiment refers to the appropriatedistance table data and obtains the appropriate distance correspondingto the display width obtained by the display width obtaining program.

FIG. 14 shows an exemplary operation of the adjustment process of theCPU 36. Upon initiation of the adjustment process, the CPU 36 obtainsthe sensor width S in a step S1, according to the sensor width obtainingprogram. In this embodiment, since the sensor width S is fixed, the readsensor width data is stored in the memory area 128.

In a step S3, the CPU 36 executes the display width calculation processaccording to the display width obtaining program. The detail of thedisplay width calculation process is shown in FIG. 15. In a step S31 inFIG. 15, the CPU 36 uses the GPU 42 to display a sensor-correspondingwidth input screen as shown in FIG. 8 on the monitor 34.

Next, in a step S33, the CPU 36 determines, based on the input data inthe input data buffer 126, whether or not there is an input fordetermining the sensor-corresponding width from the controller 22 as theinput apparatus. Specifically, it is determined whether or not there isthe data indicating the fact that the operating portion 72 of thecontroller 22 is operated in the input data. The confirmation of theinput data is continued until it is determined to be “YES” in the stepS33.

If “YES” in the step S33, the CPU 36 determines, in a step S35, whetheror not the input is of a determination button. Specifically, the CPU 36determines whether or not the A button 72 d is pressed based on theinput data. If “YES” in the step S35, the CPU 36 calculates, in a stepS37, the display width D according to the equation 1. Thesensor-corresponding width W can be obtained by detecting the number ofpixels of the image 200 in widthwise direction indicating thesensor-corresponding width when the A buttons 72 is pressed.

On the other hand, if “NO” in the step S35, the CPU 36 determines, in astep S39, whether or not the input is from the right/left directionbutton. Specifically, the CPU 36 determines, based on the input data,whether one of the right direction and the left direction of the crosskey 72 a is pressed. If “YES” in the step S39, the CPU 36 updates, in astep S41, the display of the sensor-corresponding width. Specifically,the CPU 36 increases or decreases the sensor-corresponding width W by apredetermined length according to a direction indicated by the cross key72 a, and produces the image 200 indicating the sensor-correspondingwidth W using the GPU 42, and displays the image 200 on the monitor 34.Upon completion of the step S41, the process returns to the step S33.

Upon completion of the step S37, the display width calculation processends, and the process returns to a step S5 of FIG. 14.

In the step S5 of FIG. 14, the CPU 36 determines the appropriatedistance L between the controller 22 as the input apparatus and themonitor 34 as the display according to the appropriate distancedetermination program. Specifically, the CPU 36 refers to theappropriate distance table data, and reads the appropriate distance Lcorresponding to the display width D calculated in the step S37 in apredetermined area in the data memory area 112.

In a step S7, the CPU 36 obtains the input data from the input databuffer 126. Then, in a step S9, the CPU 36 determines, based on theinput data, whether or not the two markers 340 m and 340 n are imaged.Specifically, it is determined whether or not the marker coordinate dataindicating the two markers 340 m and 340 n is included the input data.

If “YES” in the step S9, the CPU 36 executes, in a step S11, the currentdistance calculation process according to the current distancecalculation program. The detail of the current distance calculationprocess is shown in FIG. 16. In a step S51 in FIG. 16, the CPU 36calculates the distance Si between the markers in the imaged image.Specifically, as described in the explanation for FIG. 10, based on themarker coordinate data included in the input data, the distance Sibetween the coordinates of the marker images 340 m′ and 340 n′ iscalculated.

Next, in a step S53, the CPU 36 calculates the imaging range width R inthe real space. Specifically, the CPU 36 calculates the imaging rangewidth R, according to the equation 4, based on the imaged image width Riread from the optical disk 18 to a predetermined area of the data memoryarea 112, the sensor width S stored in the memory area 128, and thecalculated distance Si between the markers.

Then, in a step S55, the CPU 36 calculates the current distance Lc fromthe markers 340 m and 340 n to the controller 22. Specifically, the CPU36 calculates the current distance Lc, according to the equation 5,based on the viewing angle a of the imaging element 98 read from theoptical disk 18 to the predetermined area of the data memory area 112and the calculated imaging range width R. The calculated currentdistance Lc is stored in the memory area 132. Upon completion of thecurrent distance calculation process, the process returns to a step S13of FIG. 14.

In the step S13 of FIG. 14, the CPU 36 notifies the relation between theappropriate distance and the current distance according to thenotification program. For example, in a case where the notification isperformed by the image display, the CPU 36 produces an image includingthe reference image 202 and the adjustment image 204 as shown in FIG.11, using the GPU 42, and displays the produced image on the monitor 34.The scale factor for determining the size of the adjustment image 204 iscalculated based on the appropriate distance L determined in the step S5and the current distance Lc calculated in the step S11, and stored inmemory area 134. Therefore, the size of the adjustment image 204 ischanged based on this scale factor data. In a case where thenotification is performed by the sound output, as described above, theCPU 36 produces the audio data using the DSP 52 to output the sound fromthe speaker 34 a, or transmits the sound output instruction data to thecontroller 22 to output the sound from the speaker 104 of the controller22. Further, in a case where the notification is performed by thevibration output, as described above, the CPU 36 transmits vibrationinstruction data to the controller 22 and has the vibrator 106 of thecontroller 22 vibrate. Moreover, in a case where the notification isperformed by the light output, as described above, the CPU 36 transmitsthe lighting instruction data to the controller 22 and has the LED 76 ofthe controller 22 light. It is desirable that the notification by thesound, vibration, and light is performed for a predetermined time period(predetermined number of frames), such as every few seconds, until it isdetermined to be “YES” in a following step S15.

On the other hand, if “NO” in the step S9, the process proceeds straightto the step S15. In the step S15, the CPU 36 determines whether or notthe setting is completed. Specifically, it is determined, based on theinput data, whether or not the A button 72 d for inputting the settingcompletion is pressed during the display of the image indicating therelation between the appropriate distance and the current distance as inFIG. 11, or after the notification by the sound, vibration, and light isperformed. If “NO” in the step S15, the process returns to the step S7.On the other hand, if “YES” in the step S15, the adjustment processends.

According to this embodiment, it is possible to notify the appropriatedistance between the markers 340 m and 340 n (monitor 34) and thecontroller 22. Therefore, the player is able to operate with the optimumdistance from the display screen, where a position or a directionpointed by the controller 22 of the player on the display screen matchesa position or a direction recognized on the display screen.

In the above described embodiment, the width of the display surface ofthe monitor 34, i.e., the display width D is calculated. However, inanother embodiment, the display width D may be inputted by the player.In this embodiment, as shown in FIG. 17, the CPU 36 executes the displaywidth input process in a step S3′ instead of the step S3. For example, adisplay screen configured to enable an input of a value of the displaywidth by an operation of the operating portion 72 is displayed so thatthe display width is obtained based on the input data. Further, it ispossible to make both of the calculation of the display width based onthe obtaining of the sensor-corresponding width as shown in FIG. 8 andthe player input of the width executable.

Furthermore, in another embodiment, the display width may be obtained byhaving the player select the display size. In this embodiment, forexample, the display size input screen as shown in FIG. 18 is displayedon the monitor 34. Displayed on this input screen is a list of sizes ofthe display surfaces used for the monitor 34 available in market. Theplayer may select a display size from the list using, for example, acursor 210 by operating the controller 22, and then, may input thedisplay size by pressing a determination button 212 with the cursor 210.The cursor 210 may be moved by operating the cross key 72 a of thecontroller 22, or the cursor 210 may be displayed on a pointed positionby the controller 22. Moreover, in this input screen, the display widthmay be calculated by obtaining the sensor-corresponding width as shownin FIG. 8, and it is possible to obtain the sensor-corresponding widthby pressing a setting button 214 on the input screen in FIG. 18.

In the appropriate distance table of this embodiment, as shown in FIG.19, the display width D1, D2 . . . Dn and the appropriate distance L1,L2, . . . Ln are stored in association with display sizes K1, K2, . . .Kn. Therefore, it is possible to determine the appropriate distancecorresponding to the inputted display size by referring to this table.

FIG. 20 shows an operation of the adjustment process of this embodiment.The CPU 36 displays, in a step S101 after the step S1, the display sizeinput screen on the monitor 34. By this, the input screen as shown inFIG. 18 is displayed. Next, the CPU 36 determines, in a step S103,whether or not the setting button 214 is selected based on the inputdata. If “YES” in the step S103, the CPU 36 executes, in the step S3,the display width calculation process as described above. On the otherhand, if “NO” in the step S103, the CPU 36 executes, in a step S105, adisplay size input process. In this step S105, the display size selectedfrom the list is specified based on the input data.

Further, although the interval between the two markers 340 m and 340 nis fixed in each of the above embodiments, the marker interval may bevariable in a different embodiment. Specifically, the sensor bar 34 bmay be variable in its length, and the player may adjust the lengthaccording to the situation of each family. In this embodiment, thedistance between the markers 340 m and 340 n has to be inputted by theplayer. Therefore, in the step S1 of each of the above embodiments, forexample, the display screen configured to allow the values of the sensorwidth to be inputted by operating the operating portion 72 is displayed,and the sensor width S is obtained based on the input data. Moreover, inthe processing for determining the appropriate distance in the step S5,the appropriate distance L is calculated according to the relationobtained by the equation 2 and the equation 3.

In each of the above embodiments, the two markers are taken as anexample of the known imaging objective. However, the present inventionis not limited to the above example, and any marker whose actual sizemay be recognized may be employed. For example, a single marker of aknown size may be employed. In this case, because the size of the markeris known, it is possible to calculate the current distance between themarker and the controller 22 according to the above described methodbased on the size of the marker and the size of the marker image on theimaged image. Furthermore, if the imaging element 98 and the imageprocessing circuit 100 are capable of recognizing a color image, exampleembodiments of the present invention may be realized by setting a markerof a predetermined color and size as the known imaging objective.

Although example embodiments of the present invention has been describedand illustrated in detail, it is clearly understood that the same is byway of illustration and example only and is not to be taken by way oflimitation, the spirit and scope of the present invention being limitedonly by the terms of the appended claims.

1. A non-transitory storage medium having stored thereon a program whichupon execution by a computer performs a method of adjusting using apointing device for obtaining input data from an input apparatusprovided with an imaging unit for imaging a known imaging object, theinput data being one of data of an imaged image obtained by the imagingunit and data obtained by performing a predetermined operation on theobtained data of the imaged image, and performing information processingusing the input data to output to a display apparatus, the methodcomprising: obtaining a maximum width of a display surface of saiddisplay apparatus; determining an appropriate distance between saidinput apparatus and said display apparatus according to the obtainedmaximum width of said display surface; calculating a current distancefrom said input apparatus to said imaging object based on said inputdata; and generating a notification according to relation between saidappropriate distance and said current distance.
 2. A non-transitorystorage medium having stored thereon a program for adjusting a pointingdevice according to claim 1, wherein said notification is generated byone of outputting image display, outputting sound such as a voice soundand a sound effect, and having said input apparatus vibrate, accordingto a difference between said current distance and said appropriatedistance.
 3. A non-transitory storage medium having stored thereon aprogram for adjusting a pointing device according to claim 1, whereinsaid notification is generated by displaying, on said display apparatus,a reference image having a predetermined size and an adjustment imagevarying in size in conjunction with said current distance and becomingthe same size as said reference image when said current distance isequal to said appropriate distance.
 4. A non-transitory storage mediumhaving stored thereon a program for adjusting a pointing deviceaccording to claim 1, wherein said known imaging object is constitutedby two markers positioned with a predetermined distance therebetweenmeasured in a first scale, said two markers being positioned in avicinity of a predetermined side of the display surface of said displayapparatus so that a positioning direction of said two markers is inparallel to the predetermined side, said method further comprisesobtaining said predetermined distance either inputted by a user orpreviously stored, and said obtaining the maximum width of the displaysurface includes: measuring a distance between said two markers in asecond scale by varying an interval in the positioning direction betweensaid two markers in an image displayed on said display surface,according to the input by the user; and calculating the maximum width ofsaid display surface in said first scale based on a ratio between saidobtained predetermined distance and the measured distance between saidmarkers, as well as based on the maximum width of said display surfacein said second scale.
 5. A non-transitory storage medium having storedthereon a program for adjusting a pointing device according to claim 1,wherein said known imaging object is constituted by two markerspositioned with a predetermined distance therebetween measured in afirst scale, said two markers being positioned in a vicinity of apredetermined side of the display surface of said display apparatus sothat a positioning direction of said two markers is in parallel to thepredetermined side, said method further comprises obtaining saidpredetermined distance either inputted by a user or previously stored,and said obtaining the maximum width of the display surface includes:measuring a distance between said two markers in a second scale byvarying an interval in the positioning direction between said twomarkers in an image displayed on said display surface, according to theinput by the user; and calculating the maximum width of said displaysurface in said first scale based on a ratio between the measureddistance between said markers and the maximum width of said displaysurface in said second scale, as well as based on said obtainedpredetermined distance.
 6. A non-transitory storage medium having storedthereon a program for adjusting a pointing device according to claim 4,wherein said current distance is calculated using the distance betweensaid two markers on said imaged image.
 7. A non-transitory storagemedium having stored thereon a program for adjusting a pointing deviceaccording to claim 5, wherein said current distance is calculated usingthe distance between said two markers on said imaged image.
 8. Anon-transitory storage medium having stored thereon a program foradjusting a pointing device according to claim 1, wherein said obtainingthe maximum width of the display surface includes obtaining the maximumwidth, inputted by the user, of the display surface of said displayapparatus.
 9. A pointing device for obtaining input data from an inputapparatus provided with an imaging means for imaging a known imagingobjective, the input data being one of data of an imaged image obtainedby the imaging means and data obtained by performing a predeterminedoperation on the obtained data of the imaged image, and performinginformation processing using the input data to output to a displayapparatus, comprising: a display surface width obtaining means forobtaining a maximum width of a display surface of said displayapparatus; an appropriate distance determining means for determining anappropriate distance between said input apparatus and said displayapparatus according to the maximum width of said display surfaceobtained by said display surface width obtaining means; a currentdistance calculating means for calculating a current distance from saidinput apparatus to said imaging objective based on said input data; anda notifying means for making a notification according to relationbetween said appropriate distance and said current distance.
 10. Apointing device according to claim 9, wherein said notifying means makessaid notification by one of outputting image display, outputting soundsuch as a voice sound and a sound effect, and having said inputapparatus vibrate, according to a difference between said currentdistance and said appropriate distance.
 11. A pointing device accordingto claim 9, wherein said notifying means makes said notification bydisplaying, on said display apparatus, a reference image having apredetermined size and an adjustment image varying in size inconjunction with said current distance and becoming the same size assaid reference image when said current distance is equal to saidappropriate distance.
 12. A pointing device according to claim 9,wherein said known imaging objective is constituted by two markerspositioned with a predetermined distance therebetween measured in afirst scale, said two markers being positioned in a vicinity of apredetermined side of the display surface of said display apparatus sothat a positioning direction of said two markers is in parallel to thepredetermined side, and the pointing device further comprising: adistance obtaining means for obtaining said predetermined distanceeither inputted by a user or previously stored, wherein said displaysurface width obtaining means includes: a marker distance measuringmeans for measuring a distance between said two markers in a secondscale by varying an interval in the positioning direction between saidtwo markers in an image displayed on said display surface, according tothe input by the user; and a display surface width calculating means forcalculating the maximum width of said display surface in said firstscale based on a ratio between said predetermined distance obtained bysaid distance obtaining means and the distance between said markersmeasured by said marker distance measuring means, as well as based onthe maximum width of said display surface in said second scale.
 13. Apointing device according to claim 9, wherein said known imagingobjective is constituted by two markers positioned with a predetermineddistance therebetween measured in a first scale, said two markers beingpositioned in a vicinity of a predetermined side of the display surfaceof said display apparatus so that a positioning direction of said twomarkers is in parallel to the predetermined side, and the pointingdevice further comprising: a distance obtaining means for obtaining saidpredetermined distance either inputted by a user or previously stored,wherein said display surface width obtaining means includes: a markerdistance measuring means for measuring a distance between said twomarkers in a second scale by varying an interval in the positioningdirection between said two markers in an image displayed on said displaysurface, according to the input by the user; and a display surface widthcalculating means for calculating the maximum width of said displaysurface in said first scale based on a ratio between the distancebetween said markers measured by said marker distance measuring meansand the maximum width of said display surface in said second scale, aswell as based on said predetermined distance obtained by said distanceobtaining means.
 14. A pointing device according to claim 12, whereinsaid current distance calculating means calculates said current distanceusing the distance between said two markers on said imaged imageobtained by said imaging means.
 15. A pointing device according to claim13, wherein said current distance calculating means calculates saidcurrent distance using the distance between said two markers on saidimaged image obtained by said imaging means.
 16. A pointing deviceaccording to claim 9, wherein said display surface width obtaining meansincludes a display surface width inputting means for obtaining themaximum width, inputted by the user, of the display surface of saiddisplay apparatus.
 17. A method comprising: obtaining input data from acontroller provided with an imaging unit for imaging an imaging object;performing, by a computer system having one or more computers,information processing using the input data to generate an output to adisplay apparatus; obtaining a maximum width of a display surface ofsaid display apparatus; determining an appropriate distance between saidcontroller and said display apparatus according to the obtained maximumwidth of said display surface; calculating a current distance from saidcontroller to said imaging object based on said input data; andgenerating a notification according to relation between said appropriatedistance and said current distance.
 18. A method according to claim 17,wherein said notification is generated by one of outputting imagedisplay, outputting sound such as a voice sound and a sound effect, andhaving said controller vibrate, according to a difference between saidcurrent distance and said appropriate distance.
 19. A method accordingto claim 17, wherein said notification is generating by displaying, onsaid display apparatus, a reference image having a predetermined sizeand an adjustment image varying in size in conjunction with said currentdistance and becoming the same size as said reference image when saidcurrent distance is equal to said appropriate distance.
 20. A methodaccording to claim 17, wherein said imaging object is constituted by twomarkers positioned with a predetermined distance therebetween measuredin a first scale, said two markers being positioned in a vicinity of apredetermined side of the display surface of said display apparatus sothat a positioning direction of said two markers is in parallel to thepredetermined side, said method further comprises obtaining saidpredetermined distance either inputted by a user or previously stored,and said obtaining the maximum width of the display surface includes:measuring a distance between said two markers in a second scale byvarying an interval in the positioning direction between said twomarkers in an image displayed on said display surface, according to theinput by the user; and calculating the maximum width of said displaysurface in said first scale based on a ratio between said obtainedpredetermined distance and the measured distance between said markers,as well as based on the maximum width of said display surface in saidsecond scale.
 21. A method according to claim 17, wherein said imagingobject is constituted by two markers positioned with a predetermineddistance therebetween measured in a first scale, said two markers beingpositioned in a vicinity of a predetermined side of the display surfaceof said display apparatus so that a positioning direction of said twomarkers is in parallel to the predetermined side, said method furthercomprises obtaining said predetermined distance either inputted by auser or previously stored, and said obtaining the maximum width of thedisplay surface includes: measuring a distance between said two markersin a second scale by varying an interval in the positioning directionbetween said two markers in an image displayed on said display surface,according to the input by the user; and calculating the maximum width ofsaid display surface in said first scale based on a ratio between themeasured distance between said markers and the maximum width of saiddisplay surface in said second scale, as well as based on said obtainedpredetermined distance.
 22. A method according to claim 20, wherein saidcurrent distance is calculated using the distance between said twomarkers on an image.
 23. A method according to claim 21, wherein saidcurrent distance is calculated using the distance between said twomarkers on an image.
 24. A method according to claim 17, wherein saidobtaining the maximum width of the display surface includes obtainingthe maximum width, inputted by the user, of the display surface of saiddisplay apparatus.
 25. A system comprising: a controller having animaging element configured to image an imaging object; a computerconfigured to: obtain input data from the controller; performinformation processing using the input data to generate an output to adisplay apparatus; obtain a maximum width of a display surface of saiddisplay apparatus; determine an appropriate distance between saidcontroller and said display apparatus according to the obtained maximumwidth of said display surface; calculate a current distance from saidcontroller to said imaging object based on said input data; and generatea notification according to relation between said appropriate distanceand said current distance.
 26. A system according to claim 25, whereinsaid notification is generated by one of outputting image display,outputting sound such as a voice sound and a sound effect, and havingsaid controller vibrate, according to a difference between said currentdistance and said appropriate distance.
 27. A system according to claim25, wherein said notification is generating by displaying, on saiddisplay apparatus, a reference image having a predetermined size and anadjustment image varying in size in conjunction with said currentdistance and becoming the same size as said reference image when saidcurrent distance is equal to said appropriate distance.
 28. A systemaccording to claim 25, wherein said imaging object is constituted by twomarkers positioned with a predetermined distance therebetween measuredin a first scale, said two markers being positioned in a vicinity of apredetermined side of the display surface of said display apparatus sothat a positioning direction of said two markers is in parallel to thepredetermined side, said computer is further configured to obtain saidpredetermined distance either inputted by a user or previously stored,and said computer is further configured, so as to obtain the maximumwidth of the display surface, to: measure a distance between said twomarkers in a second scale by varying an interval in the positioningdirection between said two markers in an image displayed on said displaysurface, according to the input by the user; and calculate the maximumwidth of said display surface in said first scale based on a ratiobetween said obtained predetermined distance and the measured distancebetween said markers, as well as based on the maximum width of saiddisplay surface in said second scale.
 29. A system according to claim25, wherein said imaging object is constituted by two markers positionedwith a predetermined distance therebetween measured in a first scale,said two markers being positioned in a vicinity of a predetermined sideof the display surface of said display apparatus so that a positioningdirection of said two markers is in parallel to the predetermined side,said computer is further configured to obtain said predetermineddistance either inputted by a user or previously stored, and saidcomputer is further configured, so as to obtain the maximum width of thedisplay surface, to: measure a distance between said two markers in asecond scale by varying an interval in the positioning direction betweensaid two markers in an image displayed on said display surface,according to the input by the user; and calculate the maximum width ofsaid display surface in said first scale based on a ratio between themeasured distance between said markers and the maximum width of saiddisplay surface in said second scale, as well as based on said obtainedpredetermined distance.
 30. A system according to claim 28, wherein saidcurrent distance is calculated using the distance between said twomarkers on an image.
 31. A system according to claim 29, wherein saidcurrent distance is calculated using the distance between said twomarkers on an image.
 32. A system according to claim 25, wherein saidobtaining the maximum width of the display surface includes obtainingthe maximum width, inputted by the user, of the display surface of saiddisplay apparatus.